Traditional medical endoscopes allow the physician to see structures inside the body and, in some cases, to treat the structures, as by performing a surgical procedure or applying a drug. Traditional endoscopes use a combination of lens systems and fiber-optic cable to convey images to an eye-piece or video monitor. Because the physician can observe the structures to be treated visually, from a viewpoint on the endoscope while performing the treatment, the physician gets an immediate, easy-to-understand picture of the spatial relationships between the endoscope, surgical tools inserted through the endoscope, and the surrounding bodily structures. If the endoscope moves to a new position in the patient's body, the lens moves with it and the physician's viewpoint also moves. The physician need not make mental computations to visualize the moved location of the endoscope, or to correct the image for the changed location. In this sense, the endoscope provides a natural viewing environment. For these reasons, endoscopic procedures have been adopted widely in many fields of medicine.
However, an endoscope must include an objective lens mounted adjacent the end advanced into the patient's body, referred to as the distal end of the endoscope, and must also include either an optical system such as a fiber optic or an electronic system such as a television camera and cable for relaying the images to outside of the patient's body. The endoscope must also include a light source or light conductor for illuminating the interior of the body. Although the art has devoted considerable time and effort to miniaturization of these components, typical endoscopes are still too large and inflexible to allow their insertion into many regions of the patient's body.
Moreover, the optical images from the endoscope contain information carried by light reflected from objects and surfaces in the forward viewing volume of the endoscopic lens, i.e., the volume seen through the lens. In most applications within the body, the forward viewing volume is bounded by an opaque surface such as the surface of an organ. Stated another way, the physician looking through an endoscope can only see what is visible through the lens; the physician cannot see what lies behind the surface of the organ. Also, the field of view and viewing direction are limited by the physical characteristics of the lens and other components. Thus, medical endoscopes are most commonly used to explore the inside of cavernous organs such as the alimentary tract or lungs.
Medical imaging procedures, such as magnetic resonance imaging (MRI) and computed tomographic (CT) imaging provide noninvasive visualization of the interior of a patient. Typically, however, MRI and CT imaging is performed pre-operatively for diagnostic purposes to provide the physician with images of the patient shown in various volumetric "slices."
Volumetric information or data obtained from CT, MRI and other similar medical imaging procedures has been recently used for "virtual reality" reconstruction to allow visualization of the imaging information in a three-dimensional environment. For example, in Sherman, "Virtual Reality; A Revenue Generator for Broadband?," Communications International (1992), a surgical simulator is described that can allow the surgeon to "fly around organs" to practice operations using remote controlled robotic instruments. The article describes that pictures are taken from a small remote-controlled video camera and superimposed on X-ray or MRI scans and viewed on a monitor. As with the use of an endoscope, however, this procedure apparently calls for the use of a small video camera, and therefore would suffer from the same limitations.
Wapler & Neugebauer, "Controlling Miniature Robotic Systems in Minimally Invasive Surgery", IEEE/RSJ/GI International Conference on Intelligent Robots and Systems, v.1 (1994), states that in a so-called virtual reality surgical environment, a surgeon can observe the current position of the endoscope tip relative to the patient in a computer-generated virtual image while also viewing the direct video images obtained by real video cameras in the endoscope itself. Thus, while viewing the real image obtained by the endoscope, the physician can see a video display showing an image reconstructed from MRI or CT data, with the endoscope superimposed on the image in a position corresponding to the position of the endoscope in the body. The viewpoint of the reconstructed image can be selected arbitrarily by the physician. PCT International Publication WO 96/08209 of Visualization Technology Inc. describes other systems which detect the positions of medical instruments and provide images of the patient's body with a representation of the instrument superimposed on the image. In certain embodiments using flexible instruments and magnetic position sensor at the time of the instrument, the '209 publication also employs a separate fiber optic endoscope inserted into the body along with the instrument to track movement of the instrument tip.
In McLaurin & Jones, Jr., "Virtual Endoscope," SPIE, Vol. 2177, an endoscope usable for medical and other applications requiring three-dimensional images is described. This article predicts that "there is a future potential for attaching the scope and camera devices to telepresent robots that can be guided in a number of ways."
Wada, et al., "Three dimensional neuroimaging using computer graphics. Virtual endoscopy of the ventricular system," C I Kenkyu (Progress in Computed Imaging) (Japan) v.16:2 (1994), describes virtual endoscopic imaging, i.e., a computer graphics system which generates an image simulating the image which would be seen from an arbitrary viewpoint specified by the user. Such imaging is described as useful for preoperative image training and simulation.
Lorensen et al., The Exploration of Cross-Sectional Data with a Virtual Endoscope, in Interactive Technology and the New Paradigm for Healthcare, Morgan et al., eds. (1995), describes a similar virtual endoscopic system which works with MRI or CT data defining a three-dimensional image of a patient. Using a mouse or other computer system input techniques, the user specifies an arbitrary position within the patient, and the system reconstructs an image which would be seen from an endoscope lens disposed at that position. By specifying a series of positions along a theoretical path, the user can effectively "fly through" the data, seeing a series of images which would be seen by an endoscope moving along the theoretical path.
Virtual reality techniques as taught in Wada et al. and Loernsen et al., however, do not allow the physician to treat the patient. Thus the physician can use these techniques to study a proposed operation beforehand, and to predict what will be seen during the actual operation, but cannot use these techniques to control the actual course of an operation or other interventional procedure.
PCT International Publication WO 91/07726 of I.S.G. Technologies, Inc., and an instruction manual entitled "The Viewing Wand" describe a device using a rigid instrument with an articulated arm mounted to the patient bed. The patient's head is clamped to the bed. Angular sensors in the articulated arm detect the disposition of the instrument, and a computer system transforms this disposition into registration with previously-acquired imaging data such as MRI data. The computer system reconstructs images of the patient, such as three-dimensional images of the body or selected parts or axial, coronal or saggital "slices", with pictures of the instrument superimposed thereon. Various computer graphics techniques can be used in the reconstruction. The three-dimensional images can be made translucent or partially translucent so that the instrument can be seen in the image. The computer system can also produce a two-dimensional "trajectory" image, taken on a plane perpendicular to the axis of the instrument, looking from outside the patient, and a "perpendicular" image, showing a two-dimensional view on a cutting plane perpendicular to the axis of the instrument, centered at the tip of the instrument. In an alternate arrangement shown in the '726 publication, the instrument may be provided with a large magnetic field sensor mounted at the proximal end of the instrument, on the handle thereof, and the position and orientation of the instrument may be determined by operating the sensor to detect magnetic fields sent from a transmitter mounted in fixed position relative to the patient.
Despite all of this effort, however, there have been needs for methods and apparatus which provide the intuitive visualization and treatment capabilities of a traditional endoscope without the size and other limitations of the traditional endoscope.